Method for three-dimensional inspection using patterned light projection

ABSTRACT

A three-dimensional inspection system and method is used to obtain information about three-dimensional articles with specular surfaces having a shape and positive or negative height by projecting a pattern of light onto the articles at an oblique angle. The system includes a patterned light projector with optical axis disposed at an oblique angle with respect to the plane of the article being inspected, an extended light source, and an image detector disposed above the article to detect the image of the pattern on the article. The light pattern includes lines with a substantially equal thickness and spacing. The spacing of the lines is greater than a spacing or pitch of the specular elements. An image processor, coupled to the image detector, receives the image, locates the lines, and measures the lateral shift of the lines. Height information is determined from the lateral shift and projection angle using triangulation.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation-in-part of U.S. patentapplication No. 09/150,716 filed Sep. 10, 1998.

STATEMENT OF GOVERNMENT SUPPORT

[0002] This invention was made with Government support under ContractNo. DAAH01-96-C-R208 awarded by the Department of the Army. TheGovernment has certain rights in the invention.

TECHNICAL FIELD

[0003] The present invention relates to systems and methods forthree-dimensional inspection of articles and more particularly, to asystem and method for inspection of three-dimensional electronicpackages using a projected pattern of light.

BACKGROUND INFORMATION

[0004] Projecting patterned or structured light onto an article is awell known technique for obtaining three-dimensional informationpertaining to the article. As shown in FIG. 1, a projector 1 is used toproject a pattern of light, such as a series of parallel lines 2, ontosurfaces 4, and 6. The axis 3 of the projector 1 is oriented at an anglewith respect to these surfaces 4, and 6. When the lines 2 are projectedonto a three-dimensional surface 4 that has elements which are raised,depressed or a combination of both with respect to another surface 6,the lines 2 appear to shift laterally between these surfaces 4, and 6when viewed from above, for example, using camera 8 and monitor 9. Themagnitude of the lateral shift between the lines 2 on surfaces 4, and 6yields information about the distance between the surface 4 and thesurface 6. For example, the lateral shift between the lines 2 and theangle of projection can be used to calculate the height, positive ornegative, of the surface 4 with respect to the surface 6 usingtriangulation.

[0005] Existing systems and methods for three-dimensional inspectionusing projected light patterns, however, do not adequately provide anaccurate inspection of three-dimensional electronic packages havingspecular surfaces with a shape and height, such as Ball Grid Array (BGA)devices or lead frames used in the manufacturing of electronic devices.Accurate inspection of electronic packages and other such articlesrequires high resolution measurements of the lateral shift in the linesor pattern projected onto the article. If the projected pattern or imageis not properly focused or is distorted, measurements of the lateralshift in the lines of the projected pattern may not be accurate. In theexisting systems having an angled projector 1, the projected image maynot be in focus if the Scheimpflug condition is not satisfied, as willbe discussed in greater detail below. Blurring of the lines in theprojected pattern also typically occurs as the lines move away from thefocus of the projector 1. As a result, the width of the lines projectedonto the article may not be consistent over the entire range of thearticle being inspected. The width and spacing of the projected linescan also vary as a result of an effect commonly referred to askeystoning, as will be described in greater detail below.

[0006] Existing patterned light projectors also encounter problems as aresult of three-dimensional specular surfaces, such as the solder ballson BGA devices or the leads on lead frames used in the manufacturing ofelectronic devices. The reflection of light from specular surfaces oftencauses a saturation of pixels in the camera and necessitates the use ofcameras with high dynamic ranges or logarithmic responses. Also, if aseries of lines or a similar pattern is projected with a spacing equalto the spacing of the three-dimensional features having a shape andheight, such as solder balls on a BGA device, light will reflect betweenneighboring solder balls. This type of reflection will adversely affectthe image detected by the camera and thus will result in an inaccuratemeasurement of the shift in the lines. Furthermore, when the articlebeing inspected has a surface and three-dimensional surface objects withshapes and heights as well as different reflectivities, such as thesolder balls on the substrate of a BGA device or the leads on leadframes used in the manufacturing of electronic devices, it is difficultto view both surfaces with a single exposure without losing informationon one of the surfaces by either saturating one of the lines or causingone to be in the noise region of the signal.

[0007] Accordingly, a need exists for a system and method forthree-dimensional inspection that projects patterned light in a mannerthat reduces unwanted reflection from three-dimensional specularsurfaces, provides a projected pattern that is in focus, and allows highresolution measurements of the reflected light pattern to accuratelydetermine three-dimensional information. In particular, a need existsfor a system and method for three-dimensional inspection of BGA devicesor similar articles having raised, depressed, or a combination of raisedand depressed specular surfaces of different reflectivities.

SUMMARY OF THE INVENTION

[0008] The present invention features a system for three-dimensionalinspection of an article having at least one three-dimensional objectprotruding from or depressed into the surface of an article to beinspected and which article is supported generally in a plane. Apatterned light projector having an optical axis is disposed at anoblique angle with respect to the plane of the article. The patternedlight projector includes a light source for generating light anddirecting the light toward the article along the optical axis. A lightpatterning member is disposed between the light source and the articleat an oblique angle with respect to the optical axis, for creating thelight pattern as the light passes through the light patterning member.

[0009] The system also comprises a light pattern detector, for detectingthe light pattern reflected from the surface of the article. In oneexample, the light pattern detector includes an image detector, such asa camera, disposed generally above the article, for detecting the imageof the light pattern reflecting from the article. The image of the lightpattern is preferably in a plane parallel to the plane of the articlesuch that the light pattern projected onto the article and the articleitself are simultaneously in focus. The system also comprises an imageprocessor, for receiving the image detected by the image detector andfor processing the image to determine three-dimensional informationpertaining to the article.

[0010] The present invention also features a patterned light projectorfor use in the system for three-dimensional inspection of an article.The preferred embodiment of the patterned light projector includes anextended light source for generating light and projecting the lighttoward the article generally along the optical axis disposed at anoblique angle with respect to the plane of the article. The lightpatterning member preferably includes a pattern of lines and createslines of light projected onto the article.

[0011] The patterned light projector also includes one or moreprojection lenses disposed between the light patterning device and thearticle, for projecting the light pattern on the surface of the article.A condenser lens is also preferably disposed between the extended lightsource and the light patterning member, for providing a substantiallyuniform illumination of the light patterning member.

[0012] Examples of the extended light source include a fiber opticbundle, a light line, or an array of light emitting diodes (LEDs). Inone example, the light patterning member includes a transparent slidewith a mask forming the pattern of lines. Alternatively, the lightpatterning member includes a programmable mask.

[0013] The pattern of lines on the light patterning member preferablyhave a varying, non-uniform spacing and thickness such that the lines oflight projected onto the article at the oblique angle have asubstantially equal spacing and thickness on and along the surface ofthe article. The spacing of the lines of light is preferably greaterthan the spacing of the specular elements or objects on the articlebeing inspected, for example, the solder balls on the BGA device or theleads on lead frames used in the manufacturing of electronic devices.

[0014] The patterned light projector also preferably includes a depth offocus modifier, for providing a lower f-number (i.e. less depth offocus) in a direction along a length of the lines of light projected onthe surface of the article, and a higher f-number (i.e. greater depth offocus) in a direction along a width of the lines. This provides verycrisp, clear edges on the projected lines. In one example, first andsecond projection lenses are used with an elongated aperture disposedbetween the first and second projection lenses and oriented lengthwisewith respect to the lines.

[0015] According to one embodiment, the projector shifts the lines oflight projected onto the article. The shifting can be accomplished by amechanism for moving the projector, a rotatable transparent lightshifting plate disposed between the patterned light projector and thearticle, or a programmable mask.

[0016] The present invention also features a method of inspectingthree-dimensional features of an article having an array ofthree-dimensional specular elements having a shape and positive ornegative height. The method comprises the steps of: placing the articleon an article support such that the article generally lies in a plane;projecting lines of light onto the article, wherein a spacing of thelines of light is greater than a spacing of the three-dimensionalspecular elements such that one of the lines of light approaches a topof one of the raised specular elements or the bottom of a depressedspecular element while a consecutive one of the lines is on an oppositeside of a consecutive one of the raised or depressed three-dimensionalspecular elements; detecting at least a first image of the lines oflight projected on to the article at a first position; and processingthe first image to locate at least one of the lines of light projectedon the planar surface of the article and to measure a lateral shift ofthe one line of light at a point on one of the raised or depressedthree-dimensional specular elements, for calculating a height, positiveor negative, of the three-dimensional specular element at that point. Inone example, the article includes a BGA device having an array of solderballs disposed on a substrate such that the spacing of the lines isgreater than a pitch of the array of solder balls.

[0017] The preferred method further includes shifting the lines of lightprojected onto the article by a fraction of a projected line width to asecond position and detecting a second image of the lines of light atthe second position. The step of locating the lines includes subtractinggray scale values in the second image of the line from corresponding(i.e., same pixel) gray scale values in the first image of the line toobtain a synthetic image of the line. The synthetic image extendsthrough a zero crossing plane and includes positive pixel values abovethe zero crossing plane and negative pixel values below the zerocrossing plane. The points at which the synthetic image intersects thezero crossing plane are located and used to calculate the lateral shiftof the line.

[0018] The step of locating the points at which the synthetic imageintersect the zero crossing plane includes fitting a surface using aplurality of best fit splines to a portion of the synthetic imageproximate said zero crossing plane and determining where the best fitsplines intersect the zero crossing plane. Alternatively, a local bestfit plane can be fit to a small portion of the synthetic image proximatethe zero crossing plane.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] These and other features and advantages of the present inventionwill be better understood by reading the following detailed description,taken together with the drawings wherein:

[0020]FIG. 1 is a schematic view of a prior art system for using aprojected pattern of lines to determine three-dimensional features;

[0021]FIG. 2 is a schematic side view of a three-dimensional inspectionsystem using a projected light pattern, according to the presentinvention;

[0022]FIG. 3A is plan view of a BGA device having a pattern of lines oflight projected thereon, according to one embodiment of the presentinvention;

[0023]FIG. 3B is a plan view of a surface having three-dimensionalfeatures or elements with a negative height and having a pattern oflines of light projected thereon, according to another embodiment of thepresent invention;

[0024]FIG. 4A is a side view of a BGA device with raised and roundedspecular elements having a pattern of lines of light projected thereon,according to one embodiment of the present invention;

[0025]FIG. 4B is a side view of a three-dimensional surface with acombination of raised and depressed specular elements having a patternof lines of light projected thereon, according to one embodiment of thepresent invention;

[0026]FIG. 4C is a side view of a three-dimensional surface with roundedspecular elements having a negative height and having a pattern of linesof light projected thereon, according to one embodiment of the presentinvention;

[0027]FIG. 4D is a side view of a three-dimensional surface with acombination of raised and depressed specular elements having a patternof lines of light projected thereon, according to another embodiment ofthe present invention;

[0028]FIG. 4E is a side view of a three-dimensional surface with acombination of raised and depressed specular elements having a patternof lines of light projected thereon, according to yet another embodimentof the present invention;

[0029]FIG. 5 is a side schematic view of an arrangement of the lightpatterning member in the patterned light projector to satisfy theScheimpflug condition, according to the present invention;

[0030]FIG. 6 is a schematic representation of a pattern of lines havinga substantially equal spacing and thickness;

[0031]FIG. 7 is a schematic representation of a pattern of lines havinga varying spacing and thickness to prevent keystoning, according to oneembodiment of the present invention;

[0032]FIG. 8 is a schematic view of the elongated aperture used in thepatterned light projector to modify the depth of focus, according to thepresent invention;

[0033]FIG. 9 is a schematic representation of the elongated apertureproviding a short depth of focus along the length of the projected linesand a long depth of focus along the width of the projected lines,according to the present invention;

[0034]FIG. 10A is a diagrammatic view of a projected line image profiletaken across the width of a projected line image;

[0035]FIG. 10B is a diagrammatic view of projected line image profilesof a projected line image shifted from a first position to a secondoverlapping position, according to the method of the present invention;

[0036]FIG. 10C is a diagrammatic view of a synthetic line image profileobtained by subtracting the projected line image at the second positionfrom the projected line image at the first position, according to themethod of the present invention;

[0037]FIG. 11 is a diagrammatic view of a “best fit” spline fit to thesynthetic line image profile, according to one method of the presentinvention;

[0038]FIG. 12 is a top schematic view of synthetic line images obtainedby subtracting the projected line images taken at the first and secondoverlapping positions, according to the method of the present invention;

[0039]FIG. 13 is a three-dimensional diagrammatic view of a “best fit”surface fit to a synthetic line image having, according to anothermethod of the present invention;

[0040] FIGS. 14A-14C are diagrammatic views illustrating the apparent“shift” of the centroid of a projected line image on surfaces havingrapidly varying reflectivities;

[0041]FIG. 15A-15C are diagrammatic views illustrating the lack of a“shift” of the centroid of a synthetic line image on surfaces havingrapidly varying reflectivities, according to the method of the presentinvention; and

[0042]FIGS. 16A and 16B are schematic views of a transparent lightshifting plate used to shift the light planes and the lines projectedonto the article, according to one embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] The three-dimensional inspection system 10, FIG. 2, according tothe present invention, projects a light pattern, such as a pattern oflines, onto the surface of a three-dimensional article 12 to beinspected, such as a BGA device or lead frames used in the manufacturingof electronic devices, and analyzes the reflected light pattern todetermine three-dimensional characteristics of the surface of thearticle three-dimensional 12. In general, the three-dimensionalinspection system 10 includes an article support 14 that supports thethree-dimensional article 12 generally in a plane 16, and a patternedlight projector 20 that projects the pattern of light generally along anoptical axis 22 onto the three-dimensional article 12 with the opticalaxis 22 of the projector 20 at an oblique angle α with respect to theplane 16 of the three-dimensional article 12. A light pattern (or image)detector 24, such as a CCD camera, detects the light pattern (or image)projected onto the three-dimensional article 12, and a light pattern (orimage) processor 26 receives the reflected light pattern or imagedetected by the image detector 24 and processes the image to determinethree-dimensional information pertaining to the three-dimensionalarticle 12.

[0044] The patterned light projector 20 includes a light source 30 thatgenerates light and projects the light generally in the direction of theoptical axis 22. A light patterning member 32 is disposed between thelight source 30 and three-dimensional article 12, for creating the lightpattern as the light passes through the light patterning member 32. Acondenser lens 34 is optionally disposed between the light source 30 andlight patterning member 32. The light source 30 and condenser lens 34(when used) provide a substantially uniform illumination of the lightpatterning member 32. The patterned light projector 20 also includes oneor more projection lenses 36, 38 that project the pattern of light ontothe three-dimensional article 12 generally along the optical axis 22. Inone example, the projection lenses 36, 38 include two 100 mm Cooketriplets (such as Part No. 01LAS007 available from Melles Griot) used ina symmetrical relationship back-to-back.

[0045] The light source 30 used in the patterned light projector 20 ispreferably an extended light source, such as a large fiber optic bundle,a light line, or an array of light emitting diodes (LEDs). By using anextended light source, the light propagates from a sizable area ratherthan a single point (i.e., when using a point source). Since the lightprojected onto the three-dimensional article 12 having a shape andnegative or positive height comes from a wide range of angles, not justa single angle, this type of illumination is advantageous forthree-dimensional articles 12 having surfaces that are specular innature (e.g., solder balls, solder bumps, solder depressions or valleys)disposed on surfaces having a low reflectivity (e.g., the substrate ofBGA devices or the lead frames used in the manufacturing of electronicdevices). The light reflected from the specular surfaces on thethree-dimensional article 12 is thus more diffuse in nature as a resultof the extended light source and reduces saturation in the imagedetector or camera 24. When saturation is reduced, the need for cameraswith high dynamic ranges or logarithmic responses is eliminated.Although the exemplary embodiment in FIG. 2 shows a fiber optic typelight source 30 used with the condenser lens 34, other types ofillumination can be used and the condenser lens 34 is not needed for alltypes of illumination.

[0046] According to the exemplary embodiment, the light pattern is apattern of generally straight, parallel lines 41, FIG. 3A-B, projectedonto the three-dimensional article 12, although the present inventioncontemplates other projected patterns of light. The light passingthrough the light patterning member 32 creates light planes 40, FIGS.4A-E, that form the image of parallel lines 41 when projected onto thethree-dimensional article 12. In one embodiment, the light patterningmember 32 is a transparent slide, such as glass, with a mask formed onthe transparent slide to create the light planes 40 as the light passesthrough. Alternatively, the light patterning member 32 may include aprogrammable mask, such as a liquid crystal display (LCD).

[0047] One example of the three-dimensional article 12 to be inspectedincludes a substrate 42 having an array of raised and rounded specularelements 44, such as an array of solder balls on a BGA device. As eachline image 41 passes over the raised and rounded specular elements 44 onthe three-dimensional article 12, the line image 41 appears to shiftlaterally when viewed from above (FIG. 3A). Thus, the image of areflected line 41 a passing over raised, rounded specular elements 44includes a portion 46 reflected from the higher surface of the raised,rounded specular elements 44 and a portion 48 reflected from thegenerally lower planar surface of the substrate 42. On the raised,rounded specular elements 44, the lateral shift δ of each line portion46 varies as the height h of the raised, rounded element 44 varies.

[0048] A second example of the three-dimensional article 12 to beinspected includes a substrate 42 having an array of depressed, roundedspecular elements 122, having a negative height, such as die forelectronic components. As each line image 41 passes over the depressedand rounded specular elements 122 on the three-dimensional article 12,the line image 41 appears to shift laterally when viewed from above(FIG. 3B). Thus, the image of a reflected line 41 c passing over thedepressed and rounded specular elements 122 includes a portion 118reflected from the lower surface of the depressed and rounded specularelements 122 and a portion 120 reflected from the generally higherplanar surface of the substrate 42. On the depressed and roundedspecular elements 122, the lateral shift δ of each line portion 118varies as the negative height h1 of the depressed and rounded element122 varies. The present invention also contemplates using thethree-dimensional inspection system to inspect other types of electronicpackages or other articles having three-dimensional surfaces, includingthose pictured in FIGS. 4C and 4D. Packages having three-dimensionalfeatures having an irregular shape and a positive or negative height areconsidered to be within the scope of the present invention as well.

[0049] According to the preferred exemplary embodiment, the lines 41have a spacing that is greater than the spacing of the raised specularelements 44, i.e., the pitch of the BGA device. The lines 41 are spacedsuch that when one of the line images 41 a approaches the top of anelement 44 a, the next subsequent line 41 b is on the opposite side ofthe next subsequent element 44 b (FIGS. 3A and 4A). For example, if aBGA device has a pitch of 0.050″ and the solder balls have a diameter of0.030″, a preferred line spacing is roughly 0.075″. By providing aspacing of the line images 41 that is greater than the spacing or pitchof the raised specular elements 44 or other three-dimensional featurehaving a shape and a height, the amount of reflections between thespecular elements is reduced, preventing the degradation of lineposition measurements.

[0050] According to the preferred embodiment, the projector 20 projectsthe lines such that the image of the lines is preferably in focus in adesired plane 52 parallel to the plane 16 of the article. For inspectionof BGA devices and other similar three-dimensional devices having ashape and a positive height, the desired plane 52 is parallel to thesubstrate 42 and lies about half-way between the substrate 42 and thetops of the raised specular elements 44, such as solder balls. Fordevices having elements with a negative height, the desired plane 52lies halfway between the substrate and the bottom of the negativeelement. For devices having both elements with a positive height andelements with a negative height, the desired plane 52 would be the planeof the substrate, i.e., the plane approximately halfway between theextreme positive and negative heights. To focus the line images 41 inthis desired plane 52, the light patterning member 32, FIG. 5, ispreferably oriented at an oblique angle β with respect to the opticalaxis 22. Since the three-dimensional article 12 (FIG. 2) lies in theplane 16 having an angle α with respect to the optical axis 22, thelight patterning member 32 should also be angled with respect to theoptical axis 22 to satisfy the Scheimpflug condition. The Scheimpflugcondition is satisfied when the plane 50 of the object or lightpatterning member 32, the plane 52 of the line image, and the plane 53of the lens 38 all intersect at a common line 51. Thus, the lightpatterning member 32 is angled at the angle β with respect to theoptical axis 22 and is positioned relative to the lens 38 such that thedesired image plane 52 is parallel to plane 16 of the article 12. Theangles α and β are preferably both about 45°.

[0051] As a result of angling the light patterning member 32 withrespect to the optical axis 22, the pattern on the light patterningmember 32 will be distorted as the pattern is projected onto thethree-dimensional article 12—an effect commonly referred to askeystoning. For example, if a light patterning member 32 has a patternof equally spaced, equal thickness lines 54, FIG. 6, the light planespassing through the light patterning member at the angle β will createlines 54, FIG. 7, having a varying spacing and thickness. Thus, thelight patterning member 32 is preferably an anti-keystoning slide havinga pattern of lines with a varying spacing and thickness to produce aprojected pattern of lines on the article 12 having a substantiallyequal spacing and equal thickness. An anti-keystoning slide produces theopposite effect in that a pattern of lines having a varying spacing andthickness (as shown in FIG. 7) is provided on the light patterningmember or slide 32 to yield a projected pattern of substantially equallyspaced, equal thickness lines (as shown in FIG. 6).

[0052] One way of creating a light patterning member 32 having a patternof lines that provide anti-keystoning is to place an illuminated patternof evenly spaced equal thickness lines in the workspace on the articlesupport 14 (FIG. 2). A film plate is placed in the position of the lightpatterning member 32 and is exposed to the illuminated pattern of evenlyspaced equal thickness lines on the article support 14 through theprojection lens system 36, 38. Since the light rays through the systemare reversible, the developed plate will have a pattern of lines withthe varying spacing and thickness needed to correct the keystoningeffect. Alternatively, the process can be simulated on a computer andthe computer output can be used to print, expose, cut or otherwisegenerate a light patterning member or mask having the desiredanti-keystoning pattern.

[0053] According to the preferred embodiment, the patterned lightprojector 20 (FIG. 2) also provides a depth of focus modifier 60 thatprovides different f-numbers in the horizontal and vertical directions.The depth of focus modifier 60 is preferably an elongated aperture 61,FIG. 8, such as a narrow rectangular slit, disposed between theprojection lenses 36, 38 such that the aperture 61 is oriented in thesame direction as the lines in the projected light pattern. The aperture61 provides a lower f-number in the direction of arrows 64 along thelength of the aperture 61 (also along the length of the projected lines)and a higher f-number in the direction of arrows 66 along the width ofthe aperture 61 (also along the width of the projected lines).

[0054] For example, if the lines on the light patterning member 32 aredisposed vertically, the elongated aperture 61 disposed between thelenses 36, 38 is oriented vertically to yield a higher f-number in thehorizontal direction of the lines than in the vertical direction. In oneexample, the aperture 61 has a width in a range of about 1 mm and alength in a range of about 50 mm resulting in a greater depth of focusin one direction by a factor of 50.

[0055] The low f-number in the direction 64 along the length of thelines is important for the purpose of illuminating each point on thethree-dimensional specular objects from a wide angular distribution.This reduces the dynamic range needed by the image detector or camera 24that gathers the reflected energy from the three-dimensional specularobjects. The aperture 61 does this by reducing the “hot spot” energyreflected by a directly reflecting specular point as compared to theenergy reflected by a diffusely reflecting point. The directlyreflecting specular point sends energy from only one small portion 68 ofthe elongated aperture 61 into a lens 62 of the image detector or camera24. A diffuse point absorbs energy from the entire elongated aperture 61and reflects a portion of that energy into the camera lens 62.

[0056] The elongated aperture 61 also ensures that a greater variety ofspecular points on the three-dimensional specular elements having ashape and positive or negative height, such as the raised, roundedspecular elements 44 (e.g., solder balls) become directly reflecting andvisible to the camera lens 62. The light ray 70 from the portion 68 atthe center of the elongated aperture 61 is directed from a specularlyreflecting point 69 on a three-dimensional specular element having ashape and positive or negative height, such as raised, rounded specularelement 44 into the lens 62. Other points on the three-dimensionalspecular element having a shape and positive or negative height, such asraised, rounded specular element 44 direct light rays 72, 74 fromportions 76, 78 at the outer extent of the elongated aperture 61 intothe lens 62. As an analogy, a specular point 69 on a three-dimensionalfeature such as curved or rounded surface element 44 can be approximatedby a tiny flat mirror 71 tangent to the curved or rounded surface 44 atthat specular point 69. An image of the camera lens 62 is present in thetiny flat mirror 71. Any light rays (e.g., light ray 70 from the centerpoint 68 of the aperture 61) that pass through both the specular point69 and the image of the camera lens 62 in the tiny flat mirror 71 willbe specularly reflected into the camera lens 62. Any light rays (e.g.,light rays 72, 74 from the outer points 76, 78 of the aperture 61) thatdo not pass through the specular point 69 and the image of the cameralens 62 in the mirror 71 will be specularly reflected away from thecamera lens 62. Other specular points on the three-dimensional specularelement having a shape and positive or negative height, such as raised,curved or rounded surface 44 can be approximated by rotating the mirror71 about an axis 73, which has the effect of directing the light rays72, 74 from the outer points 76, 78 of the aperture 61 into the cameralens 62.

[0057] The low f-number of the elongated aperture 61, FIG. 9, along thelength direction 64 provides a rapid divergence and a short depth offocus from the focused spot 80, as indicated by arrows 82. The highf-number along the width direction 66 provides slow divergence and along depth of focus from the focused spot 80, as indicated by arrows 84.The high divergence and resulting short depth of focus cannot bediscerned in the direction along the length of the lines, since there isno detail to show a de-focus in that direction. Thus, as the lines moveaway from the focus of the projector 20, the lines will blur morerapidly in length than in width, thereby maintaining a substantiallyconstant line width over the entire range of the three-dimensionalarticle 12. A substantially constant line width over the entire range ofthe three-dimensional article 12 allows higher resolution measurements.Accordingly, using an elongated aperture, such as a rectangular slit, inlieu of a more commonly used circular aperture allows for a greaterdepth of focus while also allowing sufficient light to reach thethree-dimensional article 12.

[0058] The elongated aperture 61 is preferably placed at the commonfocal point of the two projection lenses 36, 38, thereby making theoverall projection lens system telecentric in one axis only. Thisprovides constant magnification in the direction transverse to theprojected lines, which has the effect of keeping the spacing between theprojected lines constant, i.e., neither converging nor diverging,regardless of the longitudinal distance from the lenses 36, 38 along theprojection axis 22.

[0059] A method of inspecting three-dimensional features of anthree-dimensional article 12 according to the present invention beginsby placing the three-dimensional article 12, such as a BGA device, onthe article support 14 beneath the image detector or camera 24 (See FIG.2). The camera 24 is preferably disposed above the three-dimensionalarticle 12 with an axis 28 of the camera 24 generally perpendicular tothe plane 16 of the article 12. Alternatively, the camera 24 can also bedisposed at an acute angle with respect to the plane 16 of thethree-dimensional article 12. The light planes 40 are then projectedonto the three-dimensional article 12 at the oblique angle α withrespect to the plane 16 of the three-dimensional article 12 to form theimage of the lines 41 (FIGS. 3 and 4).

[0060] The image of the lines 41 reflected from the three-dimensionalarticle 12 are then detected by the image detector or camera 24. In oneexample, the lens 62 of the image detector 24 includes a telecentricgauging lens. A telecentric gauging lens keeps the object constant asthe camera 24 goes in and out of focus. The whole field is viewed fromthe same perspective angle, in contrast to a standard lens where thecloser the object is, the greater the magnification. The image detector24 can be calibrated, for example, using the method of calibrating athree-dimensional sensor disclosed in U.S. Pat. No. 4,682,894,incorporated herein by reference.

[0061] The detected image is received by the image processor 26, whichlocates the line images 41 projected onto the three-dimensional article12, as described in detail below, and measures the lateral shift δ alongeach of the line images 41 (FIG. 3). Using the lateral shift δ at anygiven point on the three-dimensional specular element having a shape andpositive or negative height, such as raised specular element 44 and theprojection angle α, the positive or negative height h at that point onthe three-dimensional specular element, such as raised element 44 can becalculated by triangulation.

[0062] Each detected line image includes a series of pixels having grayscale values, which can be represented along a width of the line as aline image profile 90, FIG. 10A. Because the line image profile 90 is anirregular gaussian profile, attempting to locate the center of the lineby finding the maximum gray scale value or attempting to locate the grayscale values at the edge of the line is not adequate for high resolutionmeasurements. Thus, the preferred method of the present invention uses atechnique, sometimes referred to as “line splitting” or “planesplitting”, to locate the lines with less ambiguity than if a singleline image profile 90 is used.

[0063] According to this preferred method, each line is shifted by afraction of a line width from a first position to a second overlappingposition. First and second images of each line are taken at the firstand second overlapping positions, as represented by first and secondimage profiles 90 a, 90 b, FIG. 10B. In one example, the line thicknessis about 0.004″ and the lines are shifted by about 0.002″. The grayscale values of the second line image (represented by line image profile90 b) are subtracted from the corresponding (same pixel number) grayscale values of the first line image (represented by line image profile90 b) to obtain a computed or a synthetic line image, as represented bysynthetic line image profile 92. The synthetic line image corresponds tothe difference in gray scale values and has positive and negative grayscale values. The point 94 of zero amplitude between the resultingpositive and negative peaks in the synthetic image corresponds to animaginary light plane edge. The imaginary light plane edge is sharplydefined because the edge slope at the zero crossing point 94 is twicethe slope of the sides of the original light planes.

[0064] The zero crossing point 94 of the synthetic line images is usedto more accurately locate the line images and to measure the shift inthe line images projected onto the article. According to the preferredmethod, a “best fit” spline 96, FIG. 11, is passed through the pixelvalues in the synthetic line image profile 92. The location where thespline 96 passes through the axis is computed to provide the bestestimate of the location of the zero crossing point 94. This method ofcomputing the location of the zero crossing point is preferred becausepixel locations and pixel values are restricted to integers and noise isoften present in images.

[0065] Synthetic line images 98, FIG. 12, are computed for each of thedetected line images. The direction of the cross-hatching in FIG. 12 isindicative of areas in the synthetic line images 98 that lie above orbelow the zero crossing plane. One or more splines 96 can be positionedacross each of the synthetic line images 98. The direction of thespline(s) 96 used for the computation is preferably approximatelyorthogonal to the synthetic line images 98.

[0066] According to another preferred method, a “best fit” surface 100,FIG. 13, is fit to a plurality of splines 96 on each synthetic lightimage 98, for example, using a best fit (e.g., least square error)equation. This method then includes solving numerically for the curvethat defines the intersection of the “best fit” surface 100 with thezero plane 102. The best fit surface 100 is computed for the ribbonshaped region in the vicinity of the intersection of the synthetic lineimage 98 with the zero crossing plane 102. This ribbon shaped region maybe arbitrarily extended further above and below the zero plane 102 toencompass more pixel values into the best fit, which enhances thesmoothness of the resulting fit and reduces quantizing noise. Before thepixel values are incorporated into the best fit equation, any bad pixelvalues can be removed, for example, via standard filtering techniques,such as use of the median filter, as is well known in the field of imageprocessing.

[0067] This “plane splitting” technique is particularly useful where thearticle being inspected has a wide range of reflectivities, such as thesolder balls and substrate of a BGA device or other three-dimensionalspecular element having a shape and positive or negative height. Theintensity of a projected light plane has a cross section 104, FIG. 14A,with a centroid 106. When that projected light plane impinges on anobject with rapidly varying contrast, such as the change in reflectivityR shown in FIG. 14B, the centroid 106 of the light plane appears toshift in the image 108 reflected from the object, as shown in FIG. 14C.As a result of the low reflectance, the centroid 106 a of the image 108obtained by the camera appears to be to the left of the true position ofthe centroid 106 of the actual light plane. When the “plane splitting”technique described above is used, the zero point of the synthetic image98, FIGS. 15A-15B, does not appear to be shifted in the image detectedby the camera as a result of the difference in reflectivity. Thus, thecentroid 95 of the synthetic image 98 does not appear to be shifted as aresult of the change in reflectivity.

[0068] According to the preferred embodiment, the patterned lightprojector 20 (FIG. 2) is movable to provide the shifting of the lineimages 41 on the three-dimensional article 12. For example the patternedlight projector 20 can be disposed on a 1-axis translation stage 110that moves the projector 20 in a manner causing the line images 41 toshift. The system 10 preferably includes a projector controller 112coupled to the translation stage, for controlling movement of thepatterned light projector 20 and the shifting of the lines 41 projectedonto the three-dimensional article 12.

[0069] Alternatively, the light patterning member 32 may include aprogrammable mask, such as an LCD or other similar programmable display,that enables the projected lines to be shifted electronically, therebyeliminating the need for moving parts in the system. The programmablemask can also be used to vary the line spacing depending on the spacingof the three-dimensional specular elements having a shape and positiveor negative height, such as raised, rounded specular elements 44,thereby eliminating the need for multiple slides for different types ofthree-dimensional articles 12. Where a programmable mask lightpatterning member 32 is used, the projector controller 112 can becoupled to the programmable mask light patterning member 32 to controlthe electronic shifting of the lines and the spacing of the lines on thelight patterning member 32. This eliminates the need for the 1-axistranslation stage 110, used to move the projector.

[0070] According to a further alternative, the projector 20 (FIG. 2) caninclude a transparent light shifting plate 114 disposed between thesecond projection lens 38 and the three-dimensional article 12. Thepreferred embodiment of transparent light shifting plate 114, FIGS. 16Aand 16B, is made of a transparent material, such as glass or plastic,that causes the light traveling through the plate 114 to be refracted,resulting in a shift D of the projected light planes 40 and thus ashifting of the line images. The magnitude of the shift D is determinedby the index of refraction of the plate 114, the thickness of the plate114, and the angle 0 at which the plate 114 has been rotated withrespect to the incident light. For example, the light shifting plate 114is rotated through a first position, FIG. 16A, to an angle of +θ,causing the projected light planes 40 to shift to the right. Rotatingthe plate 114, FIG. 16B, in an opposite direction to a position at −θcauses the light planes 40 to shift in the opposite direction. For smallangles θ, the shift D is determined according to equation (1):$\begin{matrix}{D \cong {t \cdot {I\left( \frac{N - 1}{N} \right)}}} & \text{Equation~~1}\end{matrix}$

[0071] where t equals the thickness of the plate 114, I is the tangentof the angle θ, and N is the index of refraction of the material. Aservomotor, mechanical cam mechanism, or other similar mechanism can beused to rotate the light shifting plate 114 between the two positions.

[0072] In one example, the three-dimensional inspection system 10 can beused with a two dimensional inspection system having a ring light 116(FIG. 2), such as that disclosed in U.S. Pat. No. 5,943,125, U.S. Pat.No. 5,828,449 and U.S. Pat. No. 5,926,557, all assigned to the assigneeof the present application and incorporated herein by reference. The twodimensional inspection system is used to determine two dimensionalcharacteristics of a three-dimensional specular surface with at leastone feature or element having a shape and positive or negative height,such as raised, solder balls on a BGA device, leads on lead frames usedin the manufacturing of electronic devices or other elements on othertypes of articles, such as absence or presence, location, pitch, sizeand shape.

[0073] Accordingly, the three-dimensional inspection system and methodof the present invention projects a light pattern in a manner thatreduces unwanted reflection from specular surfaces, provides a projectedpattern that is in focus, and allows high resolution measurements of thereflected light pattern to determine three-dimensional information ofthe article being inspected.

[0074] Modifications and substitutions by one of ordinary skill in theart are considered to be within the scope of the present invention whichis not to be limited except by the claims which follow.

What is claimed is:
 1. A method of inspecting at least one feature of anarticle having at least one three-dimensional specular element, saidmethod comprising the acts of: projecting at least one line of lightonto said article having said at least one specular element; detectingat least a first image of said at least one line of light reflected fromsaid article at at least a first position; and processing said at leasta first image to locate said at least one line of light projected onsaid article and to measure a lateral shift of said at least one line oflight at a point on said at least one specular element, for calculatingat least a height of said at least one three-dimensional specularelement at said point of said lateral shift.
 2. The method of claim 1,wherein said article includes a plurality of specular elements spacedapart on said article, and wherein said act of projecting at least oneline of light onto said article having said at least one specularelement comprises projecting a plurality of lines of light onto saidarticle, wherein a spacing of said plurality of lines of light projectedonto said article is greater than said spacing of said plurality ofspecular elements such that one of said lines of light approaches a topof one of said plurality of specular elements while an adjacent one ofsaid lines is on an opposite side of an adjacent one of said pluralityof specular elements.
 3. The method of claim 2 wherein said articleincludes a ball grid array (BGA) device having an array of solder ballsdisposed on a substrate, wherein said spacing of said plurality of linesis greater than a pitch of said array of solder balls.
 4. The method ofclaim 1 further comprising, after the act of detecting said at least afirst image of said at least one line of light at said at least a firstposition, the acts of: shifting said at least one line of lightprojected onto said article by a fraction of a width of said at leastone line of light to a second position; and detecting a second image ofsaid at least one line of light reflected from said article at saidsecond position; and wherein locating said at least one line of lightincludes the acts of: subtracting gray scale values in said second imageof said at least one line of light from corresponding gray scale valuesin said first image of said at least one line of light to obtain asynthetic image of said at least one line of light, wherein saidsynthetic image extends through a zero crossing plane and includespositive pixel values above said zero crossing plane and negative pixelvalues below said zero crossing plane; and locating points at which saidsynthetic image intersects said zero crossing plane, wherein said pointsat which said synthetic image intersects said zero crossing plane areused to calculate said lateral shift of said at least one line of light.5. The method of claim 4 wherein said act of locating said points atwhich said synthetic image intersects said zero crossing plane includesfitting a plurality of best fit splines to a portion of said syntheticimage proximate said zero crossing plane and determining where said bestfit splines intersect said zero crossing plane.
 6. The method of claim 4wherein said act of locating said points at which said synthetic imageintersects said zero crossing plane includes fitting a best fit plane toa portion of said synthetic image proximate said zero crossing plane anddetermining where said best fit plane intersects said zero crossingplane.
 7. The method of claim 1 wherein said at least one line of lightis projected onto said article such that said at least one line is infocus in a plane generally parallel to a plane of respose of saidarticle.
 8. The method of claim 1 wherein projecting said at least oneline of light onto said article includes providing a lower f-number in adirection along a length of said at least one line of light projectedonto said article and a higher f-number in a direction along a width ofsaid at least one line of light projected onto said article.
 9. Themethod of claim 1 wherein said three-dimensional specular elementincludes a positive height.
 10. The method of claim 1 wherein saidthree-dimensional specular element includes a negative height.
 11. Amethod of inspecting three-dimensional features of an article having anarray of three-dimensional elements having a shape and negative height,said method comprising the steps of: placing an article on an articlesupport such that said article generally lies in a plane; projectinglines of light onto said article having said array of three-dimensionalelements having a shape and negative height, wherein a spacing of saidlines of light is greater than a spacing of said three-dimensionalelements having a shape and negative height such that one of said linesof light approaches a bottom of one of said three-dimensional elementshaving a shape and negative height while a consecutive one of said linesis on an opposite side of a consecutive one of said three-dimensionalelements having a shape and negative height; detecting at least a firstimage of said lines of light reflected from said article at at least afirst position; and processing said at least a first image to locate atleast one of said lines of light projected on said article and tomeasure a lateral shift of said at least one of said lines of light at apoint on one of said three-dimensional elements having a shape andnegative height, for calculating a negative height of said one of saidthree-dimensional elements having a shape and negative height at saidpoint of said lateral shift.
 12. The method of claim 11 wherein saidarticle includes an array of three-dimensional elements having a shapeand negative height disposed on a substrate, wherein said spacing ofsaid lines is greater than a pitch of said array of three-dimensionalelements having a shape and negative height.
 13. The method of claim 11further including, after the step of detecting said at least a firstimage at said at least a first position, the steps of: shifting saidlines of light projected onto said article by a fraction of a line widthto a second position; detecting a second image of said lines of lightreflected from said article at said second position; and whereinlocating said at least one of said lines of light includes the steps of:subtracting gray scale values in said second image of said at least oneof said lines from corresponding gray scale values in said first imageof said at least one of said lines to obtain a synthetic image of saidat least one line, wherein said synthetic image extends through a zerocrossing plane and includes positive pixel values above said zerocrossing plane and negative pixel values below said zero crossing plane;and locating points at which said synthetic image intersects said zerocrossing plane, wherein said points at which said synthetic imageintersects said zero crossing plane are used to calculate said lateralshift of said at least one of said lines.
 14. The method of claim 13wherein said step of locating said points at which said synthetic imageintersects said zero crossing plane includes fitting a plurality of bestfit splines to a portion of said synthetic image proximate said zerocrossing plane and determining where said best fit splines intersectsaid zero crossing plane.
 15. The method of claim 13 wherein said stepof locating said points at which said synthetic image intersects saidzero crossing plane includes fitting a best fit plane to a portion ofsaid synthetic image proximate said zero crossing plane and determiningwhere said best fit plane intersects said zero crossing plane.
 16. Themethod of claim 11 wherein said lines are projected onto said articlesuch that said lines are in focus in a plane generally parallel to saidplane of said article.
 17. The method of claim 11 wherein projectingsaid lines onto said article includes providing a lower f-number in adirection along a length of said lines of light projected onto saidarticle and a higher f-number in a direction along a width of said linesof light projected onto said article.
 18. A method of inspectingthree-dimensional features of an article having an array of acombination of three-dimensional elements having a shape and positiveand negative heights, said method comprising the steps of: placing anarticle on an article support such that said article generally lies in aplane; projecting lines of light onto said article having said array ofthree-dimensional elements having a shape and negative height, wherein aspacing of said lines of light is greater than a spacing of saidthree-dimensional elements having a shape and negative height such thatone of said lines of light approaches a bottom of one of saidthree-dimensional elements having a shape and negative height while aconsecutive one of said lines is on an opposite side of a consecutiveone of said three-dimensional elements having a shape and negativeheight; detecting at least a first image of said lines of lightreflected from said article at at least a first position; and processingsaid at least a first image to locate at least one of said lines oflight projected on said article and to measure a lateral shift of saidat least one of said lines of light at a point on one of saidthree-dimensional elements having a shape and negative height, forcalculating a negative height of said one of said three-dimensionalelements having a shape and negative height at said point of saidlateral shift.
 19. The method of claim 18 wherein said article includesan array of a combination of three-dimensional elements having a shapeand negative and positive heights disposed on a substrate, wherein saidspacing of said lines is greater than a pitch of said array of acombination of three-dimensional elements having a shape and negativeand positive heights.
 20. The method of claim 18 further including,after the step of detecting said at least a first image at said at leasta first position, the steps of: shifting said lines of light projectedonto said article by a fraction of a line width to a second position;detecting a second image of said lines of light reflected from saidarticle at said second position; and wherein locating said at least oneof said lines of light includes the steps of: subtracting gray scalevalues in said second image of said at least one of said lines fromcorresponding gray scale values in said first image of said at least oneof said lines to obtain a synthetic image of said at least one line,wherein said synthetic image extends through a zero crossing plane andincludes positive pixel values above said zero crossing plane andnegative pixel values below said zero crossing plane; and locatingpoints at which said synthetic image intersects said zero crossingplane, wherein said points at which said synthetic image intersects saidzero crossing plane are used to calculate said lateral shift of said atleast one of said lines.
 21. The method of claim 20 wherein said step oflocating said points at which said synthetic image intersects said zerocrossing plane includes fitting a plurality of best fit splines to aportion of said synthetic image proximate said zero crossing plane anddetermining where said best fit splines intersect said zero crossingplane.
 22. The method of claim 20 wherein said step of locating saidpoints at which said synthetic image intersects said zero crossing planeincludes fitting a best fit plane to a portion of said synthetic imageproximate said zero crossing plane and determining where said best fitplane intersects said zero crossing plane.
 23. The method of claim 18wherein said lines are projected onto said article such that said linesare in focus in a plane generally parallel to said plane of saidarticle.
 24. The method of claim 18 wherein projecting said lines ontosaid article includes providing a lower f-number in a direction along alength of said lines of light projected onto said article and a higherf-number in a direction along a width of said lines of light projectedonto said article.